High speed microscope with spectral resolution

ABSTRACT

A system and method of high-speed microscopy using a two-photon microscope with spectral resolution. The microscope is operable to provide two- to five-dimensional fluorescence images of samples, including two or three spatial dimensions, a spectral dimension (for fluorescence emission), and a temporal dimension (on a scale of less than approximately one second). Two-dimensional (spatial) images with a complete wavelength spectrum are generated from a single scan of a sample. The microscope may include one of a multi-beam point scanning microscope, a single beam line scanning microscope, and a multi-beam line scanning microscope. The line scans may be formed using one or more of curved mirrors and lenses. The multiple beams may be formed using one of a grating, an array of lenses, and a beam splitter.

RELATED APPLICATIONS

The present application claims priority to U.S. Provisional Application61/472,761, filed Apr. 7, 2011, and of U.S. Provisional Application61/480,083, filed Apr. 28, 2011, the entire contents of each of whichare hereby incorporated by reference.

BACKGROUND

Embodiments of the present invention relate to microscopes with spectralresolution.

SUMMARY

Laser scanning microscopes (such as two-photon and confocal microscopes)enable acquisition of images of narrow sections of cells and tissuesthat emit light in response to receiving laser energy. The images mayinclude one or more spatial dimensions (e.g., x, y, and z dimensions).Multiple images may be stitched together or otherwise combined to createthree-dimensional (3-D) images. In some instances, images are capturedover a period of time to add a temporal dimension to the data collected.For example, images acquired over a period of time may be viewed insequence to illustrate changes over time. In some instances, in additionto one or more spatial and temporal dimensions, a spectral dimension ofthe light emitted from the sample is obtained. Such spectral informationprovides various advantages, such as enabling the detection offluorescence from multiple spectral variants of the samples' tags.

Embodiments of the present invention provide a system and method forhigh-speed microscopy with spectral resolution. Embodiments include atwo-photon microscope with spectral resolution providing four- orfive-dimensional fluorescence images of samples, including two or threespatial dimensions, a spectral dimension (for fluorescence emission),and a temporal dimension (on a scale from less than one second to aboutsixty seconds). Embodiments enable, via a single scan, generation of 2-Dor 3-D (spatial) images for a complete wavelength spectrum.

High speed acquisition of spectrally resolved images of a sample enablesthe study of highly dynamic samples emitting energy (e.g., fluorescence,transmitted light, elastically scattered light (i.e., Raman scatteredlight), second and third harmonic generation, etc.) at multiplewavelengths of interest and a reduction in the time to study manysamples in series. Furthermore, due to the speed with which photons canbe collected and pixel data output by the camera, the laser power and,thereby, photon flux of the excitation beam, can be significantlyincreased. Additionally, providing a wide range of spectral resolutionvia a single scan for each sample voxel avoids the need for filterchanges or multiple scans of the sample, which slow sample scanningtimes.

With the use of a combination of simultaneous multi-color analysis andrapid scanning methods detailed herein, the speed of analysis of asingle slide can be reduced to a few minutes or less. In a researchsetting, the collection of data is accelerated and more rapid analysison more samples can be performed. In turn, researchers are able toperform their work more rapidly and collect more data for statisticalrelevance. Embodiments of the invention enable cellular and molecularbiologists, biochemists, and other life-scientists to investigatedynamic features of multiple protein populations, includingco-localization and protein-complex formation and trafficking, andligand-induced changes in conformation and oligomeric status.

In one embodiment, the invention provides a microscope for generating amulti-dimensional, spectrally resolved image. The microscope includes alight source that generates a pulsed light beam, a curved mirror thatreflects the pulsed light beam as a light beam line, a scanningmechanism, a dispersive element, and a detector. The scanning mechanismscans the light beam line across a sample to cause the sample to emitenergy. The dispersive element receives the emitted energy from thesample and disperses the emitted energy into spectral components thatform continuous spectrum area, the continuous spectrum areacorresponding to the light beam line. The detector receives thecontinuous spectrum.

Embodiments of the microscope further include a second curved mirrorthat reflects the pulsed light beam as a second light beam line. Thescanning mechanism scans the second light beam line across the sample tocause the sample to emit additional energy. The dispersive elementreceives the additional emitted energy from the sample and disperses theadditional energy into second spectral components that form a secondcontinuous spectrum area, the second continuous spectrum areacorresponding to the second light beam line. The detector receives thesecond continuous spectrum.

In another embodiment, the invention provides a multi-beam line scanningmicroscope for generating a multi-dimensional, spectrally resolvedimage. In this configuration, the microscope provides an acquisitionspeed higher than that of a single beam line scanning microscope by afactor equal to the number of lines, because each line of the multi-beamline scanning microscope scans a smaller area than the area scanned by asingle line in a single line scanning microscope. The microscopeincludes a light source that generates a pulsed light beam, a multi-beamgenerator that receives the pulsed light beam and emits multiple lightbeams, a light beam line generator, a scanning mechanism, a dispersiveelement, and a detector. The light beam line generator receives themultiple light beams and emits multiple light beam lines. The scanningmechanism simultaneously scans the multiple light beam lines across asample to cause the sample to emit energy. The dispersive elementreceives the emitted energy from the sample and disperses the emittedenergy into spectral components that form continuous spectrum areas,each continuous spectrum area corresponding to one of the multiple lightbeam lines. The detector receives each continuous spectrumsimultaneously.

In another embodiment, the invention provides a multi-beam pointscanning microscope for generating a multi-dimensional, spectrallyresolved image. The microscope includes a light source that generatespulsed light beam, a multi-beam generator that receives the pulsed lightbeam and emits multiple light beams, a scanning mechanism, a dispersiveelement, and a detector. The scanning mechanism simultaneously scans themultiple light beams across a sample, the sample emitting energycorresponding to each of the multiple light beams. The dispersiveelement receives the emitted energy from the sample and disperses theemitted energy into spectral components that form continuous spectrumlines, each continuous spectrum line corresponding to one of themultiple light beams. The detector receives each continuous spectrumline simultaneously.

In some embodiments, the multi-beam generator is one of a grating, abeam splitter, and an array of lenses. In some instances, the light beamline generator includes a first curved mirror and a second curvedmirror, or a first lens and a second lens.

In some embodiments, the scanning mechanism includes computer-controlledx-y scanning mirrors positioned between the light source and the sampleand/or computer-controlled sample positioners that move the sample tocause the multiple light beams to scan the sample. In some embodiments,the scanning mechanism further descans the energy emitted from thesample before the energy emitted from the sample reaches the detector,and the detector has a generally planar detection surface with a lengthand a width at least half the size of the length. The detector has afirst axis along the length of the surface and corresponding to aspatial dimension of the sample, and a second axis along the width ofthe surface and corresponding to frequency.

In some embodiments, the microscope further includes an imaging modulefor receiving pixel data output by the detector and generating one ormore images based on the pixel data. In some embodiments, the microscopeis a two-photon microscope. Additionally, the curved mirror may be adeformable mirror shaped by control signals from a mirror controller. Insome embodiments, the emitted energy is one of fluorescence, elasticallyscattered light, second harmonic signals, and third harmonic signals. Insome embodiments, the microscope further includes a first and secondobjective. The first objective is positioned between the scanningmechanism and the sample, and receives the scanning light (e.g., lightbeam point(s) or light beam line(s)) from the scanning mechanism andtransmits the scanning light to the sample. The second objective ispositioned between the sample and the dispersive element, and receivesthe emitted energy from the sample and transmits the emitted energy tothe dispersive element.

In some embodiments, the multiple light beams include at least a firstlight beam with a first wavelength, and a second light beam having asecond wavelength. The first light beam and the second light beam followthe same path when scanned across the sample. Additionally, the firstlight beam and the second light beam alternate between leading andtrailing positions along the path. The first light beam is scannedacross the sample in an x-direction at a y-position, and the secondlight beam is simultaneously scanned across the sample in thex-direction at the y-position, the second light beam being offset in thex-direction from the first light beam. Additionally, the continuousspectrum line for the first light beam and the continuous spectrum linefor the second light beam follow the same path across the detectorduring the scan. The continuous spectrum line for the first light beamand the continuous spectrum line for the second light beam alternatebetween leading and trailing positions along the path.

In another embodiment the invention provides a method of generating amulti-dimensional, spectrally resolved image. The method includesgenerating a pulsed light beam, reflecting, by a curved mirror, thepulsed light beam as a light beam line, a scanning step, a dispersingstep, and a receiving step. In the scanning step, the light beam isscanned across a sample to cause the sample to emit energy. In thedispersing step, the emitted energy from the sample is dispersed intospectral components that form a continuous spectrum area, the continuousspectrum area corresponding to each point along the light beam line. Inthe receiving step, a detector receives the continuous spectrum.

The method may further include reflecting, via a second curved mirror,the pulsed light beam as a second light beam line; scanning the secondlight beam line across the sample to cause the sample to emit additionalenergy; dispersing the additional energy into second spectral componentsthat form a second continuous spectrum area, the second continuousspectrum area corresponding to each point along second light beam line;and receiving, by the detector, the second continuous spectrum.

The method may also include outputting, by the detector, pixel dataresulting from receiving the continuous spectrum and the secondcontinuous spectrum. The pixel data is received by an imaging module.The imaging module then generates one or more images based on the pixeldata.

In another embodiment, the invention provides a method of generating amulti-dimensional, spectrally resolved image. The method includesgenerating a pulsed light beam and receiving, via a multi-beamgenerator, the pulsed light beam. The multi-beam generator emitsmultiple light beams with round cross-sections in response to the pulsedlight beam. The multiple light beams are focused by an objective todiffraction limited spots (or points) which are scanned across a sample,and the sample emits energy corresponding to each of the multiple lightbeams. The emitted energy is dispersed into spectral components thatform continuous spectrum lines, with each continuous spectrum linecorresponding to one of the multiple light beams. A detector receiveseach continuous spectrum line simultaneously.

In some embodiments, the detector outputs pixel data resulting fromreceiving the continuous spectrum lines, which is received by an imagingmodule. The imaging module then generates one or more images based onthe pixel data.

Other aspects of the invention will become apparent by consideration ofthe detailed description and accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a microscope system according to embodiments of theinvention.

FIG. 2 illustrates a multi-beam point scanning microscope with spectralresolution.

FIGS. 3A-B illustrate a multi-beam point scan on a sample and detector,respectively.

FIG. 4 illustrates a single-beam line scanning microscope with spectralresolution.

FIG. 5 illustrates a curved mirror generating a beam line.

FIGS. 6A-B illustrate a single-beam line scan on a sample and detector,respectively.

FIG. 7 illustrates a multi-beam line scanning microscope with spectralresolution.

FIGS. 8A-B illustrate a multi-beam line scan on a sample and detector,respectively.

FIGS. 9A-E illustrate various shapes for curved mirrors used inembodiments of the invention.

FIGS. 10A-B illustrate a deformable mirror and associated controller.

FIGS. 11A-C illustrate various multi-beam generates used in embodimentsof the invention.

FIG. 12 illustrates a half-descanned, multi-beam point scanningmicroscope with spectral resolution.

FIG. 13 illustrates a descanned, multi-beam point scanning microscopewith spectral resolution.

FIG. 14 illustrates a half-descanned, multi-beam point scanningmicroscope with spectral resolution.

FIG. 15 illustrates a descanned multi-beam line scan on a narrow-fieldcamera.

FIGS. 16A-B illustrate a high resolution binning technique.

FIGS. 17A-B illustrate a high speed binning technique

FIGS. 18A-B illustrate techniques for sorting and trapping cells to bescanned.

FIG. 19 illustrates a method of scanning and analyzing one or moresamples to detect emissions of one or more particular wavelengths.

FIG. 20 illustrates scanning areas of a sample undergoing a two-stagescan.

FIG. 21 illustrates a high speed microscope with transmission imaging.

FIG. 22 illustrates a high speed microscope with excitation beams havingvarious wavelengths.

FIG. 23A-D illustrates a sample receive excitation beams having variouswavelengths.

FIG. 24A-D illustrates a detector receive emitted energy from a sampleexcited using beams with various wavelengths.

FIG. 25A-B illustrates a sample and detector of a high speed microscopehaving pairs of excitation beams having various wavelengths.

DETAILED DESCRIPTION

Before any embodiments of the invention are explained in detail, it isto be understood that the invention is not limited in its application tothe details of construction and the arrangement of components set forthin the following description or illustrated in the following drawings.The invention is capable of other embodiments and of being practiced orof being carried out in various ways. Also, it is to be understood thatthe phraseology and terminology used herein is for the purpose ofdescription and should not be regarded as limited. The use of“including,” “comprising,” or “having” and variations thereof herein ismeant to encompass the items listed thereafter and equivalents thereofas well as additional items. The terms “mounted,” “connected,” and“coupled” are used broadly and encompass both direct and indirectmounting, connecting and coupling. Further, “connected” and “coupled”are not restricted to physical or mechanical connections or couplings,and can include electrical connections or couplings, whether direct orindirect. Also, electronic communications and notifications may beperformed using any known means including direct connections, wirelessconnections, etc.

It should be noted that a plurality of hardware and software baseddevices, as well as a plurality of different structural components maybe utilized to implement the invention. Furthermore, and as described insubsequent paragraphs, the specific configurations illustrated in thedrawings are intended to exemplify embodiments of the invention and thatother alternative configurations are possible.

FIG. 1 illustrates a two-photon microscope system 100 including acontroller 102, memory 104, and user input/output (I/O) 106 coupledtogether via bus 107. The controller 102 may include one or more of ageneral purpose processing unit, a digital signal processor, a fieldprogrammable gate array (FPGA), an application specific integratedcircuit (ASIC), and other processing devices operable to carry out thefunctions attributable to the controller 102 and described herein. Thememory 104 may store instructions executed by the controller 102 tocarry out the aforementioned functions, may store data for thecontroller 102, such as image data, and may load data to the controller102, such as program data, calibration data, etc. for use by thecontroller 102 during operation of the microscope system 100.

The user I/O 106 enables a user to interact with microscope system 100.For instance, the user I/O 106 includes a display 108 for displaying agraphical user interface to enable a user to control the microscopesystem 100 and to display images generated by the microscope system 100.The user I/O 106 may further include input devices (e.g., a mouse,keyboard, etc.) and, in some instances, the display 108 is a touchscreen display capable of receiving user input. In some embodiments, themicroscope system 100 further includes a communication module (notshown) for communicating with remote devices. For instance, thecommunication module may enable the microscope system 100 to communicateover a local area network (LAN), wide area network (WAN), the Internet,etc., via one or more of wired and wireless connections. Accordingly,images generated by the microscope system 100 may be shared with userson remote devices, stored on remote servers (e.g., on a “cloud”), sentto nearby portable devices (e.g., a smart phone, tablet, personalcomputer, laptop, etc.) of a local user.

The microscope system 100 further includes a light source 110, one ormore mirror(s) 112, sample positioners 114, and a detector 116 (alsoreferred to as a camera) in communication with the controller 102,memory 104, and user I/O 106 via bus 107. The controller 102 controlsthe light source 110 to generate light that is directed by the mirror(s)112 towards a sample positioned by the sample positioners 114. Thesample emits energy (e.g., light) towards the detector 116. The detector116 may be, for instance, an electron multiplying charge coupled devicethat includes an array of pixels, a CMOS camera, a 2-D array ofphotomultiplier tubes, or another type of detector that includes atwo-dimensional array of pixels. The detector outputs a digital signalor signals (i.e., pixel data) to the controller 102 representative ofthe intensity of light from the sample that impinges the array ofpixels. The controller 102 includes an imaging module 118 thatinterprets the data from the detector 116 and forms an image of thearray of the sample being investigated. The controller 102 furtherincludes an analysis module 120 for analyzing the pixel data from thedetector 116 and/or images formed by the imaging module 118.

FIG. 2 illustrates the microscope system 100 implemented as a multi-beampoint scanning microscope 130 for analyzing sample 132. Forsimplification, the controller 102, memory 104, user I/O 106, and bus107 are not shown. The microscope 130 includes a pump laser 134 and apulsed laser 136 for emitting a pulsed beam of light 138 towards atelescope 139. The telescope 139 expands and transmits the pulsed beamof light 138 towards a multi-beam generator 140. The pump laser 134 maybe a high power solid state laser operating at five watts to providecontinuous wave (CW) light at a wavelength of 532 nanometers (nm),although other pump lasers 134 and other laser outputs may be used insome embodiments. The pulsed laser 136 may be a mode-locked TI:Sapphirelaser that generates femtosecond pulses of near-infrared light (centeredat approximately 800 nm with a bandwidth of a few nm to 120 nm). In someembodiments, the pulsed laser 135 is a different laser type and/orgenerates light having a different wavelength and bandwidth. In someembodiments, light generating devices different from the pump laser 134and pulsed laser 136 are used to generate light for the microscope 130.

The multi-beam generator 140 converts the pulsed beam of light 138 intomultiple beams 142. In FIG. 2, the multi-beam generator 138 is shown togenerate three beams 142. Generally, the multiple beams 142 are formedto be substantially equivalent (e.g., in intensity, wavelength, etc.).Exemplary multi-beam generators 138 are described in greater detail withrespect to FIG. 11A-C. The multiple beams 142 are reflected by themirror 144 towards a pair of computer-controlled x-y scanning mirrors(“scanning mirrors”) 146, including mirrors 146 a and 146 b. The “x” and“y” of the x-y scanning mirrors refer to orthogonal directions on thesample 132. In some embodiments, the scanning mirrors 146 are a pair ofmirrors attached to galvanometric scanners having a 10 mm aperture.

The multiple beams 142 reflected by the scanning mirrors 146 arereceived by a lens 147. The lens 147 focuses and transmits the multiplebeams 142 to a dichroic mirror 150. The dichroic mirror 150 is a shortpass mirror that reflects light having a long wavelength, but allowslight having a short wavelength to pass through. Accordingly, the longwavelength portions of the multiple beams 142 transmitted from the lens147 are reflected towards lens 148. In some embodiments, a long passmirror that reflects light having a short wavelength, but allows lighthaving a long wavelength to pass through. The lens 148 collimates andtransmits the light beams 142 to an objective lens 152. The objectivelens 152 focuses each point of the multiple beams 142 to a uniquediffraction-limited spot (i.e., point) of the sample 132. The spots aregenerally at the same x-dimension location on the sample 132, but spacedapart in the y-dimension. Using diffraction limited beam points (spots)helps avoid photo bleaching of areas of the sample not being scanned atthat moment.

As each of the diffraction limited spots is scanned across the sample132 in the x-direction by the scanning mirrors 146, the sample emitsfluorescence beams 154 back to the objective lens 152. The fluorescencebeams 154 are transmitted by the objective lens 152 to the lens 148. Thelens 148 focuses each of the emitted fluorescence beams 154 to aparticular point on an electron multiplying charge-coupled device(detector) 158. The fluorescence beams 154, however, first pass throughthe short-pass dichroic mirror 150 and a light dispersive element 160.The short-pass dichroic mirror 150 allows visible light to pass through,while reflecting most of the infrared light components of the emittedfluorescence beams 154. The dispersive element 160 disperses the lightinto its spectral components to form a continuous spectrum of varyingwavelengths that spread in the y-direction of the detector 158.Accordingly, the fluorescence beams 154, after passing through thedispersive element 160, reach the detector 158 as three wavelengthspectra extending along the y-direction. Each wavelength spectrumimpinges the detector 158 at the same x-position, but is spaced apart inthe y-direction.

In some instances, an additional short pass filter (not shown) isprovided between the dichroic mirror 150 and the dispersive element 160to further eliminate residual infrared components of the emittedfluorescence beams 154 not filtered by the dichroic mirror 150, whichcould otherwise overwhelm the visible components of interest on thedetector 158.

In the above embodiments, the sample 132 is assumed to be positioned ona static platform while the impinging light is scanned. In someembodiments, rather than scanning the multiple beams 142 across thesample 132 using scanning mirrors 146, sample positioners 162 (e.g.,nanopositioners) are used to move the sample while the scanners 132 and,therefore, the multiple beams 142, remain in a static position. In someembodiments, to perform a complete area scan of the sample 132, thescanning mirrors 146 scan the multiple beams 142 in a single direction(e.g., the x- or y-direction), while the sample positioners 162 are usedto move the sample in the other direction (e.g., the other of the x- ory-direction). The sample positioners 162 and the static platform standmay both be referred to as a sample holder. Furthermore, the sampleholder may include an automatic loading system to enable the scanning ofa plurality of samples, one after another. For instance, the sampleholder may include an automatic slide changer that is loaded withsamples 132 and that moves the samples, one after another, into positionfor scanning the samples serially.

FIG. 3A illustrates a multi-beam point scan on the sample 132 using themicroscope 130. FIG. 3A depicts the beam points 142 a-c in theirrespective starting positions at the beginning of a scan. The beampoints 142 a-c are spaced apart in the y-direction on the sample 132.The beam points 142 a-c, synchronously follow their respective scanpaths 170 a-c. The scan paths 170 a-c sweep across the sample 132 in thex-direction, increment in the y-direction, sweep back across in thex-direction, and so on, until the scan completes. A single scan of thesample 132 is completed when each portion in the scan area of the sample132 has been impinged by one of the beam points 142 a-c. Accordingly,the single scan is complete when the beam point 142 a reaches the startposition of the beam point 142 b, the beam point 142 b reaches the startposition of the beam point 142 c, and the beam point 142 c reaches theend of the scan area, which generally occur simultaneously.

FIG. 3B illustrates the resulting emitted fluorescence beams 154 a-c onthe detector 158. As noted above, the emitted fluorescence beams 154pass through a dispersive element 160 causing each point to be dispersedinto its spectral components forming spectra 172 a-c on the detector,one for each emitted fluorescence beam 154 a-c. The spectra 172 a-c havea line shape extending along the y-direction, and each is positioned atthe same x-position and spaced apart in the y-direction. Each spectrum172 is formed of a continuous spread of the emitted fluorescence over arange of wavelengths. Each point along the y-axis of the spectrum 172includes components of the emitted fluorescence at a differentwavelength. For instance, the upper/top portion of the spectrum 172 mayinclude the larger wavelengths components, while the lower/bottomportion includes the shorter wavelength components. The spectrum 172 isa continuous spectrum. The spectra 172 a-c generally follow the samepaths 170 a-c as their corresponding beam points 142 a-c. Pixel data isobtained from the detector 158 as the spectra 172 a-c reach each pointalong the scan paths 170 a-c, which corresponds to the beam points 142a-c reaching each positions of the sample 132 along scan path 170 a-c.

In a single beam point scan implementation, only a single beam point(e.g., 142 a) is generated and only a single spectrum (e.g., 172 a)results on a detector. Accordingly, the path 170 of the single beampoint and single spectrum traverses the entire sample area. In contrast,in the multi-beam scanning microscope 130, the sample area is morequickly scanned because, rather than a single beam point, three beampoints 142 a-c cover the same area. Assuming three beam points 142 a-c,the time for the multi-beam scanning microscope 130 to scan the sample132 is effectively performed in one-third the time of a single beampoint scan. Accordingly, assuming a 10 to 60 second scan for a singlebeam point scan, a multi-beam scanning microscope 130 with three beampoints would reduce the scan time to between approximately 3 to 20seconds. The number of beam points may be reduced to two or increasedbeyond three. However, the beam points 142 should remain spaced apartenough such that the resulting spectra 172 do not overlap on thedetector in the y-direction.

FIG. 4 illustrates the microscope system 100 implemented as a singlebeam line scanning microscope 200 for analyzing sample 132. Forsimplification, the controller 102, memory 104, user I/O 106, and bus107 of FIG. 1 are not shown. In FIG. 4, elements similar to those ofFIG. 2 are similarly numbered and perform similar functions, unlessotherwise noted or necessitated by differences in the respectivemicroscopes. In contrast to multi-beam point scanning microscope 130,single beam line scanning microscope 200 includes a curved mirror 202and does not include a multi-beam generator 140. Accordingly, the beam138 emitted from the pulsed laser 136 is received by the curved mirror202, which reflects the beam 138 as a beam line 204.

FIG. 5 illustrates how the curved mirror 202 reflects a beam of light asa line. As shown in FIG. 5, the light beam 138 having a generallycircular or cylindrical cross section is received by the curved mirror202 at a reflection area 206. The reflection of the light beam 138 is“stretched” such that an elongated oval is formed. The oval retainssubstantially the same cross-sectional width as the light beam 138, butthe length is increased such that the reflection has, effectively, aline shape. While other devices are capable of converting a beam 138into a beam line 204, the curved mirror 202 provides certain advantages.For example, a cylindrical lens may be used to convert the beam 138 intothe beam line 204; however, the lens produces a line with lower axialresolution. For instance, the lens includes certain chromaticaberrations that reduce the axial resolution.

Returning to FIG. 4, the beam line 204 is reflected towards the x-yscanning mirrors 146, which reflect the beam line 204 towards a lens147. As described below with respect to FIG. 6A, the beam line 204 isscanned in the y-direction on the sample, but not the x-direction.Accordingly, one of the x-y scanning mirrors may remain stationaryduring a beam line scan of the sample. The lens 147 collimates the beamline 204 and transmits the beam line 204 to the dichroic mirror 150. Thedichroic mirror 150 reflects the beam line 204 to the lens 149, which,along with the objective lens 152, focuses the beam line 204 on thesample 132 as a diffraction-limited line. The diffraction-limited lineis scanned across the sample 132 in the y-direction by the scanningmirrors 146, and the sample 132 emits a fluorescence beam line 210 backto the objective lens 152 and lens 149. The objective lens 152 may be aninfinity-corrected high numerical aperture objective, an F-Theta lens,or another focusing device. F-Theta lenses are designed to provide aflat field at the image plane of the scanning system, which isparticularly beneficial for line scanning.

The fluorescence is transmitted by the lens 149 through the short-passdichroic mirror 150 and through the lens 211 (e.g., a tube lens). Thelens 211 focuses the fluorescence beam line 210 to a line on thedetector 158. Before reaching the detector 158, however, the emittedfluorescence beam line 210 passes through the light dispersive element160. The dispersive element 160 disperses the fluorescence beam line 210into its spectral components to form a continuous spectrum of varyingwavelengths that spread in the y-direction of the detector 158.Accordingly, the fluorescence beam line 210, after passing through thedispersive element 160, reaches the detector 158 as an area withwavelength spectra extending along the y-direction and the x-dimensionof the area corresponding to the x-dimension of the sample 132.

FIG. 6A illustrates a single-beam line scan on the sample 132 using themicroscope 200. FIG. 6A depicts the beam line 204 in its startingposition at the beginning of a scan. During a scan, the beam line 204moves in the y-direction along the scan path 212. A single scan iscomplete when the beam line 204 reaches the uppermost desired y-positionof the sample 132 along the scan path 212.

FIG. 6B illustrates the resulting emitted fluorescence beam line 210 onthe detector 158. The area spectrum 214 generally follows the same path212 as the corresponding beam line 204. As noted above, the emittedfluorescence beam line 210 passes through a dispersive element 160causing the beam line 210 to be dispersed into its spectral componentsforming an area spectrum 214 on the detector 158. The area spectrum 214has a rectangular shape extending along the x- and y-direction. They-direction corresponds to wavelength, and the x-direction correspondswith the spatial, x-dimension of the sample 132. Accordingly, each pointalong the x-direction of the area spectrum 214 has a continuous spreadof the emitted fluorescence over a range of wavelengths in they-direction of the detector 158. The area spectrum 214 is associatedwith the (spatial) y-position of the sample 132 currently receiving thebeam line 204. Pixel data is obtained from the detector 158 as the areaspectrum 214 reaches each y-position, which corresponds to the beam line204 reaching each y-positions of the sample 132 along scan path 212.

In the single-beam line scan implementation, the scan speed is improvedrelative to a single beam point scan in that the line covers thex-dimension of the sample 132 to be scanned, and the beam line 204 ismerely moved along the y-dimension of the sample 132. As such, eachpoint along the x-dimension of the sample 132 is excited simultaneouslyby the beam line 204. In a single-beam line scan implementation, thespeed of the detector 152 may become the limiting factor for the time tocomplete a scan. That is, the scan will be as fast as the detector 158can convert received fluorescence and export corresponding pixel data,which, with current technology, results in a scan time of a sample beingunder 10 seconds. The scan time may be further improved with fasterdetector technology.

FIG. 7 illustrates the microscope system 100 implemented as a multi-beamline scanning microscope 250 for analyzing sample 132. Forsimplification, the controller 102, memory 104, user I/O 106, and bus107 of FIG. 1 are not shown. In FIG. 7, elements similar to those ofFIGS. 2 and 4 are similarly numbered and perform similar functions,unless otherwise noted or necessitated by differences in the respectivemicroscopes.

In the multi-beam line scanning microscope 250, the beam 138 emittedfrom the pulsed laser 136 is received by two curved mirrors 252 a and252 b, which reflect the beam 138 as a beam lines 254 a and 254 b,respectively. The beam lines 254 a-b are reflected towards the x-yscanning mirrors 146, which reflect the beam lines 254 a-b towards thelens 147. The lens 147 collimates the beam lines 254 a-b and transmitsthe beam lines 254 a-b to the dichroic mirror 150. The dichroic mirror150 reflects the beam lines 254 a-b to the lens 149 and the objectivelens 152, which focus the beam lines 254 a-b on the sample 132 asdiffraction limited lines. The diffraction limited lines are parallelwith each other along the x-dimension of the sample 132 and spaced apartin the y-dimension of the sample. The diffraction limited lines arescanned across the sample 132 in the y-direction by the scanning mirrors146, and the sample 132 emits fluorescence beam lines 256 a-b back tothe objective lens 152 and lens 149. The emitted fluorescence beam lines256 a and 256 b correspond to the beam lines 254 a and 254 b,respectively. The objective lens 152 may be an infinity-corrected highnumerical aperture objective, an F-Theta lens, or another focusingdevice. As noted above, the F-Theta lens is particularly beneficial forline scanning.

The fluorescence is transmitted by the objective lens 152 and lens 149through the short-pass dichroic mirror 150 and through the lens 211. Thelens 211 focuses the fluorescence beam lines 256 a-b to correspondinglines on the detector 158. Before reaching the detector 158, however,the emitted fluorescence beam lines 256 a-b pass through the lightdispersive element 160. The dispersive element 160 disperses thefluorescence beam lines 256 a-b into their spectral components to form acontinuous spectrum of varying wavelengths that spread in they-direction of the detector 158. Accordingly, the fluorescence beamlines 256 a-b, after passing through the dispersive element 160, reachthe detector 158 as areas with wavelength spectra extending along they-direction and the x-direction corresponding to the x-dimension of thesample 132.

FIG. 8A illustrates a multi-beam line scan on the sample 132 using themicroscope 250. FIG. 8A depicts the beam lines 254 a-b in theirrespective starting position at the beginning of a scan. During a scan,the beam lines 254 a-b move in the y-direction along the scan paths 258a-b, respectively.

FIG. 8B illustrates the resulting emitted fluorescence beam lines 254a-b on the detector 158. The area spectra 260 a-b generally follow thesame paths 258 a-b as the corresponding beam lines 254 a-b. As notedabove, the emitted fluorescence beam lines 256 a-b pass through adispersive element 160 the beam lines 256 a-b to be dispersed into itsspectral components forming area spectra 260 a-b on the detector 158.The area spectra 260 a-b have rectangular shapes extending along the x-and y-direction. The y-direction corresponds to wavelength, and thex-direction corresponds with the spatial, x-dimension of the sample 132.Accordingly, each point along the x-direction of the area spectra 260a-b has a continuous spread of the emitted fluorescence over a range ofwavelengths in the y-direction of the detector 158. Pixel data isobtained from the detector 158 as the area spectra 260 a-b reach eachy-position, which corresponds to the beam lines 256 a-b reaching eachy-positions of the sample 132 along scan path 258 a-b.

In the multi-beam line scan implementation, the scan speed is improvedrelative to a single and multi-beam point scans in that the lines 254 aand 254 b cover the x-dimension of the sample 132 to be scanned. Thatis, each point along the x-dimension of the sample 132 at a firsty-position is scanned simultaneously by the beam line 254 a, and eachpoint along the x-dimension of the sample 132 at a second y-position isscanned simultaneously by the beam line 254 b. To complete a singlescan, the beam lines 254 a-b are merely moved along the y-dimension ofthe sample 132 along paths 258 a-b.

In the multi-beam line scan implementation, the scan speed is improvedrelative to a single beam line scan in that two beam lines 256 a and 256b simultaneously cover different regions of the sample 132, which areoffset along the y-dimension of the sample. See, for example, FIGS. 8A-Bcompared with FIGS. 6A-B. As the microscope 250 includes two beam lines254 a and 254 b, a scan of the sample 132 is approximately twice as fastas a scan with a single beam line (e.g., via microscope 200 of FIG. 4).Accordingly, similar to a single-beam line scan implementation, thespeed of the detector 152 may become the limiting factor for the time tocomplete a scan.

Although the microscope 250 is shown in FIG. 7 with two curved mirrors252 each generating a beam lines 254, in some embodiments, additionalcurved mirrors 252 are provided to generate additional beam lines 254.The additional beam lines 254 are used to further increase the speed ofa scan of a sample 132. In general, the areas 260 resulting from themultiple beam lines 254 should not overlap in the y-direction on thedetector 158.

In some embodiments, the multi-beam line microscope 250 generatesmultiple beam lines 256 using alternative techniques. For example, insome embodiments, the microscope 250 includes a multi-beam generator,similar to the multi-beam generator 140. The multi-beam generator ispositioned to receive the light 138 and emit multiple beams. One or morecurved mirror(s) 252 receive the beams emitted from the multi-beamgenerator and the curved mirror(s) 252 reflect each received beam togenerate corresponding beam lines 254.

The curved mirrors 252 may also be referred to as beam line generators.In some embodiments, curved mirrors 252 are replaced with other beamline generators. For instance, the beam line generators may include oneor more lenses to receive the wide beam 138 and generate multiple beamlines, or to receive multiple beams from the multi-beam generator 140and generate multiple beam lines.

The curved mirrors 208 and 252 may have various shapes. For example,FIGS. 9A-E illustrate cross-sections of curved mirrors 208 and 252having various shapes. FIG. 9A-C illustrates the curved mirrors 208 and252 as a circularly curved mirror 280, an elliptical mirror 282, and aparabolic mirror 284, respectively. FIG. 9D illustrates the curvedmirrors 208 and 252 as a curved mirror 286 having a non-uniform curve. Alight wave front is generally not smooth or uniform. Rather, the lightwave front includes aberrations that, cause, for instance, the light tonot land directly on a surface as desired. The imperfections in thecurve of the curved mirror 286 help compensate for the aberrations of alight wave front. FIG. 9E illustrates the curved mirrors 252 as a mirrorunit 288 having multiple mirror curves 288 a-e. The mirror unit 288 maybe a single, integrated unit made of a continuous material. In someinstances, the mirror unit 288 includes individual curved mirrors (i.e.,288 a-e) mechanically coupled together to form an array, or may consistof a single deformable mirror in which individual actuators areconfigured such that the whole deformable mirror surface resembles acylindrical mirror or an array of smaller mirrors.

FIGS. 10A-B illustrate a controlled deformable mirror 290 coupled to amirror controller 292, which may be used to implement the curved mirrors208 and 252. The deformable mirror 290 includes an array of reflectivesurfaces 294 on top of actuators 296. The actuators 296 are controlledby the output signals from the mirror controller 292 to adjust the shapeof the deformable mirror 290. For example, in FIG. 10A, the controller292 outputs signals to the actuators 296 to control the deformablemirror 290 to have a flat, planar shape. In contrast, in FIG. 10B, themirror controller 292 controls the actuators 296 to control thedeformable mirror 290 to have a curved shape. The deformable mirror 290may be controlled to have any of the shapes in FIGS. 9A-E, as well asother shapes. Accordingly, the deformable mirror 290 may be selectivelycontrolled to alter the scanning of a microscope system. For instance,in the microscope 200, the deformable mirror 290 may be controlled to bethe curved mirror 202 to perform a beam line scan. Additionally, thedeformable mirror 290 may be controlled to be a flat mirror to cause themicroscope 200 to perform a beam point scan (see, e.g., FIGS. 1 and 2),or to be a series of curved mirrors to cause the microscope to performmulti-beam line scanning (e.g., FIG. 7).

In some embodiments, the mirror controller 292 is further coupled to alight sensor 298 for detecting the incident light reflected by thedeformable mirror 290. The light sensor 298 provides feedback to thecontroller 292 to more precisely control the deformable mirror 290 toproduce the desired light reflections. For instance, the deformablemirror 290 may be controlled to produce a non-uniform mirror such asillustrated in FIG. 9D. The feedback from the light sensor 298 enablesthe mirror controller 292 to adjust the deformable mirror 290 to producea desired reflection that compensates for aberrations in the light wave299.

FIGS. 11A-C illustrates various multi-beam generators that may be usedto implement multi-beam generator 140. FIG. 11A illustrates a multi-beamgenerator 300 including a beam splitter 302 and mirror 304. Themulti-beam generator 300 receives a light beam 306 a from a light source308. The light beam 306 a, with initial power (P), is transmitted to themulti-beam generator 300. The light beam 306 a is received and reflectedby the mirror 304. The beam splitter 302 receives the reflected lightbeam 306 a and reflects a light beam 306 b having power (P)=0.9·P. Theremaining 10% of P of the light beam 306 a passes through the beamsplitter 302 as light beam 310 a. The light beam 306 b is received andreflected by the mirror 304. The beam splitter 302 receives thereflected light beam 306 b and reflects a light beam 306 c havingP=0.8·P. The remaining 10% of P of the light beam 306 b passes throughthe beam splitter 302 as light beam 310 b. The light beam 306 c isreceived and reflected by the mirror 304. The beam splitter 302 receivesthe reflected light beam 306 c and reflects a light beam 306 d havingP=0.7·P. The remaining 10% of P of the light beam 306 c passes throughthe beam splitter 302 as light beam 310 c. Accordingly, the multi-beamgenerator 300 receives a single light beam with power P, and outputsthree light beams each having power 10% of P. The multi-beam generator300 may have the mirror 304 and beam splitter 302 extended further toproduce additional light beams or reduced to produce fewer beams.

FIG. 11B illustrates a multi-beam generator 320 having an array oflenses 322 receiving a single, wide light beam 324. Each lens 322focuses a portion of the wide light beam 324 to a point 326 a-e. In someembodiments, more or fewer lenses 322 are provided to alter the numberof beam points generated by the multi-beam generator 320.

FIG. 11C illustrates a multi-beam generator 330 having an opticalgrating 332 that receives a single light beam 334. The optical grating332 is a diffractive element designed to produce five light beams 336a-e of approximately equal intensities. In some embodiments, the opticalgrating 332 is designed to produce more or fewer light beams 336 a. Inother embodiments, the multi-beam generator 330 includes a diffractiveelement formed of one or more prisms, a spatial light modulator, oranother device that can produce multiple beams through a process ofdiffraction.

The microscopes 130, 200, and 250 are described and illustrated above asnon-descanned microscopes. In non-descanned microscopes, thefluorescence emitted from a sample does not pass back through the x-yscanning mirrors. Rather, the fluorescence proceeds to the detector 158without being “descanned” by scanning mirrors. Accordingly, the emittedfluorescence is scanned across the detector 158 following essentiallythe same path as the light beams scanning the sample. See, for example,FIGS. 3A-B, 6A-B, and 8A-B.

In a descanned microscope, the fluorescence emitted from the samplepasses back through the x-y scanning mirrors before reaching the camera.Accordingly, the emitted fluorescence remains stationary on the camera.For image reconstruction, the positions of the x-y scanning mirrors aremonitored such that the microscope system is able to associate emittedfluorescence with particular locations of the sample being scanned. In ahalf-descanned microscope (also referred to as half non-descannedmicroscope), the emitted fluorescence passes through one of the x-yscanning mirrors, but not both. Accordingly, the emitted fluorescence isstatic in one of the x-dimension and y-dimension on the camera, but isscanned across the camera in the other of the x-dimension andy-dimension.

Each of the microscopes 130, 200, and 250 may be implemented as adescanned microscope or half-descanned microscope. For example, FIGS.12-14 illustrate multi-beam line scanning microscopes, such asmicroscope 130, implemented as one of a descanned microscope orhalf-descanned microscope. In FIGS. 12-14, elements similar to those ofFIG. 2 are similarly numbered and perform similar functions, unlessotherwise noted or necessitated by differences in the respectivemicroscopes.

FIG. 12 illustrates the microscope system 100 implemented as amulti-beam point scanning microscope 350 having a half-descannedarrangement. For simplification, the controller 102, memory 104, userI/O 106, and bus 107 of FIG. 1 are not shown, and. In contrast to thenon-descanned microscope 130, the half-descanned microscope 350 includesx-y scanning mirrors 352 having a dichroic (short-pass) scanning mirror354 and a standard scanning mirror 356. Accordingly, the light beams 142for scanning the sample 132 are reflected by the dichroic scanningmirror 354 towards the other scanning mirror 356 on route to the sample132. The fluorescence 154 emitted by the sample 132, however, passesthrough the dichroic scanning mirror 354 towards the lens 156,dispersive element 160, and detector 158. The microscope 350 furtherincludes a mirror 358 for reflecting the light from the telescope 148 tothe objective lens 152.

FIG. 13 illustrates the microscope system 100 implemented as amulti-beam point scanning microscope 370 having a descanned arrangement.For simplification, the controller 102, memory 104, user I/O 106, andbus 107 of FIG. 1 are not shown. In the descanned microscope 370, thedichroic (long pass) mirror 150 is positioned between the multi-beamgenerator 140 and the x-y scanning mirrors 146. Accordingly, the lightbeams 142 for scanning the sample 132 are passed through the dichroicmirror 150 towards the scanning mirrors 146 on route to the sample 132.The fluorescence 154 emitted by the sample 132, however, is reflected bythe dichroic scanning mirror 150 towards the lens 156, dispersiveelement 160, and detector 158.

FIG. 14 illustrates the microscope system 100 implemented as amulti-beam point scanning microscope 380 having a half-descannedarrangement. For simplification, the controller 102, memory 104, userI/O 106, and bus 107 of FIG. 1 are not shown. In the half-descannedmicroscope 380, the dichroic (short pass) mirror 150 is positionedbetween the scanning mirrors 146 a and 146 b. The dichroic mirror 150remains static and is not scanned, in contrast to the dichroic mirror354 of FIG. 12. Accordingly, the light beams 142 for scanning the sample132 are passed through the dichroic mirror 150 towards the scanningmirrors 146 b on route to the sample 132. The fluorescence 154 emittedby the sample 132, however, is reflected by the dichroic mirror 150towards the lens 156, dispersive element 160, and detector 158. Sincethe fluorescence 154 is only scanned across the detector 158 by one ofthe scanning mirrors (scanning mirror 146 b), the fluorescence 154 isscanned across the detector 158 in one of the x- and y-dimensions, butnot both the x- and y-dimensions.

As noted above, the single beam line scanning microscope 200 and themulti-beam line scan microscope 250 may be implemented as a descannedmicroscope or half-descanned microscope. For instance, for a descannedor half-descanned single beam line scanning microscope, the flat mirror144 of FIGS. 12-14 may be replaced with the curved mirror 202. For adescanned or half-descanned multi-beam line scanning microscope, theflat mirror 144 of FIGS. 12-14 may be replaced with one or more curvedmirrors 202 or other beam point to beam line converters. Also, aspreviously noted, the curved mirrors 202 may be deformable mirrors (see,e.g., FIGS. 10A-B) or non-deformable mirrors.

A standard camera includes a pixel array that is square-shaped, such asa 512×512 pixel array. Generally, pixel data for the entire array istransmitted for each image of the camera. Such a camera may be used asthe detector 158 in embodiments of the above-noted microscopes. Thedetector 158 includes a generally planar detection surface including anarray of detector elements (i.e., pixels) that convert energy (e.g.,light) into electrical signals for output to an imaging device (e.g.,controller 102 and/or memory 104). The electrical signals may then beinterpreted, combined, filtered, and/or organized to generate an image.The electrical signals for each pixel may include a digital encoding,such as a series of binary bits, which represent characteristics of thelight received by the particular pixel. The electrical signals output bythe pixel array may be referred to collectively as “pixel data.” Thetime to transmit pixel data from the pixel array to another device(e.g., the controller 102) is a function of the number of pixels in thearray. As the pixel array size increases, the time to transmit the pixeldata increases. The time to transmit the pixel data can be a speedlimiting factor for scanning using the above-noted microscopes.Accordingly, reducing the pixel data transmission time may improvemicroscope scanning speed.

FIG. 15 illustrates a narrow detector 400 overlaid on a standard,square-shaped pixel array 402. The narrow detector 400 includes a width404 along the y-dimension and a length 406 along the x-dimension. Astandard, square-shaped pixel array 402 includes a width 408 along they-dimension and the same length 406 along the x-dimension as the narrowdetector 400. The width 408 is substantially equal to the length 406.The width 404 of the narrow detector 400, however, is significantly lessthan the length 406. For instance, the width 404 may be half of thelength 406, a third of the length 406, a quarter of the length 406, aneighth of the length 406, a sixteenth of the length, or other sizes.Generally, the lower size limit of the width 404 is constrained by thesize of the wavelength spectra in the y-dimension of the beam lines 410a-c plus the additional spacing needed between the camera boundaries andthe beam points or lines to prevent light from missing the camera andfrom overlap. For instance, in the case of the narrow detector 400illustrated in FIG. 15, the lower limit of the width 404 is the sum ofthe width of the lines 412 and the spacings 414.

The narrow detector 400, therefore, has significantly fewer pixels thana square-shaped pixel array 402. Accordingly, the time to transmit pixeldata from the narrow detector 400 to the controller 102 or memory 104 issignificantly less than the time to transmit pixel data from a camerahaving a square-shaped pixel array.

In some embodiments, the narrow detector 400 is constructed tophysically include a narrow array of pixels as described above. However,in some embodiments, the narrow detector 400 is implemented by ignoringthe additional pixels in an area 416 of the square-shaped pixel array402, or by configuring the narrow detector 400 to not output the pixeldata from the pixels in the area 416. In the case of ignoring the pixelsarea 416, the narrow detector 400 and associated microscope may beconfigured to initiate a new image capture before the output of theundesired pixel data of the pixel area 416 completes, but after thedesired pixel data from the narrow detector 400 is received.

The narrow detector 400 may be used as detector 158 in theabove-described descanned microscope 370 and half-descanned microscopes350 and 380 because the beam lines or points received by the detector158 are static in the y-dimension. For instance, the imprint of amulti-beam line scan on the narrow detector 400 is illustrated in FIG.15. The lines 410 a-c remain static over the course of a scan of thesample 132, and are not scanned along the detector 158. The staticposition of the lines 410 a-c contrasts with the non-descannedimplementations, such as shown in FIG. 8B. Additionally, in the beampoint scanning embodiments of the half-descanned microscope 380, thebeam points are scanned in the x-direction of the narrow detector 400and remain static in the y-direction. Accordingly, the pixel data fromthe pixel area 416 may be unnecessary in these implementations.

FIG. 16A illustrates an ideal imprint of a wavelength spectrum 450received by an array of pixels 452 caused by a beam point emitted from asample, such as one of the fluorescence beam points 154 on the detector158. As shown, the wavelength spectrum 450 occupies a single pixelcolumn 454, which corresponds to a particular x-position of a sample(e.g., sample 132). The wavelength spectrum 450 occupies a plurality ofrows 456 along the y-axis, each row corresponding to one or moredistinct wavelengths. For instance, row 456 a may correspond to theshortest wavelength(s), and row 456 j may correspond to the longestwavelength(s).

However, an actual imprint 458 of the wavelength spectrum 450 on thepixel array 452 spreads over into neighboring pixel columns; this spreadis usually caused by the point spread function of the microscope, whichis an intrinsic property of imaging systems. An image generated basedonly on the light received by column 454 may be less accurate in thatthe image does not represent all of the light received by the pixelarray 452 for a particular x-position of the sample. Additionally, lightoriginating from the same column in the sample, which should ideally beprojected onto wavelength spectrum 450, is projected onto adjacentcolumns within the actual imprint 458, thereby introducing image blur atthose columns.

FIG. 16B illustrates a first binning technique to address the spread ofthe wavelength spectrum 450 along the x-axis and to result in moreaccurate images. For each wavelength range (i.e., row 456), a bin 460 isused to cover multiple pixel columns. In the FIG. 16B example, each bin460 includes five pixel columns. The value attributed to each row 456a-j of the wavelength spectrum 450 is the sum of the energy received bythe pixels within each respective bin 460 a-j. Accordingly, theeffective pixel size is greater in the x-dimension than it is in they-dimension (e.g., five pixels wide by one pixel long). The sum of thelight received by one of the bins 460 is attributed to a credited pixel(or pixels) 462, which is less than the total number of pixels making upthe bin 460. Stated another way, the size of the bin 462 in thex-dimension (corresponding to the x-direction of the sample) is largerthan the size of the credited pixel(s) 462 in the x-dimension. Forinstance, the bin 460 a includes five pixels, and the light received bythe bin of five pixels is summed and, for purposes of generating animage, attributed to one credited pixel 462 a. In some embodiments, thenumber of pixels for each bin 460 may be adjusted, e.g., based on theactual spread of the wavelength spectrum 450. For instance, each bin 460may be three pixels wide, ten pixels wide, etc. Additionally, the numberof pixels making up the credited pixel(s) 462 may be more than onepixel.

Although the binning technique of FIG. 16B is illustrated with a singlebeam point scan, the binning technique may also be implemented with amultiple beam point scan. In the multi-beam point scan implementation, aseries of the bins 460 is used for each beam point (i.e., for eachwavelength spectrum) that is received by the detector 452.

The binning techniques of FIG. 16B may be implemented in software,hardware, or a combination thereof using various components of thesystem 100. For instance, the imaging module 118 of the controller 102may receive data for each pixel of the detector and sum the valuesaccording to the bins. Additionally, the imaging module 118 may be usedto configure the detector 116 such that the pixels of each bin 460 aretied together. The bin 460 of tied together pixels then output asingular data value representative of the cumulative intensity of lightreceived by the particular bin 460. Such an arrangement reduces thesignal spread and increases image contrast as, in the instance of FIG.16B, a single signal from the whole bin 460 a is credited to a singlepixel (the credited pixel 462 a).

The concept of binning is also applicable in the y-dimension of thedetector 452, as illustrated in FIGS. 17A-B. FIGS. 17A-B illustrate theidealized wavelength spectrum 450 on the row 454 of the detector 452.FIG. 17A illustrates a non-binning technique in which each pixel 470receiving the wavelength spectrum 450 is associated with one or moreunique wavelengths. Accordingly, pixel 470 a receives wavelength λ₁,while pixel 470 a receives wavelength λ₂. λ₁ and λ₂ may be wavelengthranges, rather than a particular wavelength, where λ₁ and λ₂ do notoverlap. Each pixel 470 is treated separately and the associatedelectrical signals of each pixel 470 are output by the detector 452 as aparticular data value representing the amount of light emitted from thesample at an associated wavelength (e.g., λ₁ or λ₂). Thus, in theexample of FIG. 17A, nine data values total are output by pixels 470a-j.

In FIG. 17B, a binning technique in the y-dimension is illustrated. Thepixels 470 along the row 454 are combined into bins 474, and each bin474 is associated with the wavelengths of the pixels 470 making up thebins 474. For example, bin 474 a receives wavelengths λ₁₋₃, because thepixels 470 a-c make up the bin 474 a, and the pixels 470 a-c receivewavelengths λ₁₋₃, respectively. The electrical signals of each bin 474are output by the detector 452 as a particular data value representingthe amount of light received by the combination of pixels 470 of theparticular bin 474. In the example of FIG. 17B, three data value totalare output by pixels 470 a-j, one for each bin 474 a-c. Thus, the amountof data output by the detector 452 is reduced to one third of the dataoutput by the non-binning technique shown in FIG. 17A. Accordingly, thetime to transmit pixel data from the detector 452 using the binningtechnique of FIG. 17B is significantly less than the non-binningtechnique of FIG. 17A. As noted above, the time to transmit the pixeldata can be a speed limiting factor for scanning using the above-notedmicroscopes. Accordingly, reducing the pixel data transmission time mayimprove microscope scanning speed. In some embodiments more or fewerthan three pixels 470 make up each bin 474, and more or fewer than threebins per x-position on the detector 452 are used.

Although the binning technique of FIG. 17B is illustrated with a singlebeam point scan, the binning technique may also be implemented with amultiple beam point scan, a beam line scan, and a multiple beam linescan. For instance, in the beam line scan, the wavelength spectrum 450extends along the x-axis, corresponding with various spatial positionsalong the x-axis of the sample. Accordingly, for each x-position on thedetector, one or more bins 474 of pixels receive light and output data.

Although the detector 452 speed is increased, the wavelength resolutionis reduced using the binning technique of FIG. 17B. That is, in FIG.17A, nine data points over the area of the wavelength spectrum 450 areprovided, one per pixel 470. In contrast, only three data points areprovided over the same area of the sample using the wavelength binningtechnique illustrated in FIG. 17B.

The binning techniques of FIG. 17B may be implemented in software,hardware, or a combination thereof using various components of thesystem 100. For instance, the imaging module 118 may be used toconfigure the detector 116 such that the pixels of each bin 474 are tiedtogether. The bins 474 of tied together pixels then output a singulardata value representative of the cumulative intensity of light receivedby the particular bin 474. Such an arrangement reduces the datatransmission time as described above. Although the system 100 could bearranged such that the imaging module 118 receives data for each pixelof the detector 452, and then sums the values according to the bins,this approach would generally not reduce the number of datatransmissions or the time to transmit the data from the detector 452.

The above described microscope systems may be used to implement themethods described below for scanning a sample and identifying particularcharacteristics of the sample, and for scanning a series of samples andidentifying one or more samples within the series of samples thatincludes the particular characteristics. Detecting samples or portionsof samples that emit, scatter, or transmit light at particularwavelengths is useful in identifying samples with or without aparticular makeup. For example, a sample may be tagged, through variousmethods, such as with particular chemical agents. If the sample has aparticular makeup, when the sample receives light from the microscope,the tag causes the sample to fluoresce light at particular wavelengths.If the sample does not have the particular makeup, the tag will notcause the sample to fluoresce light at the particular wavelengths.Additionally, in some instances, detecting that a sample does not emit,scatter, or transmit light at a particular wavelength is useful. Forinstance, a tagged sample that emits fluorescence at a first wavelength,and not a second wavelength, may indicate the makeup of the sample.

The microscopes described herein are operable to provide a completespectrum from a single scan. A complete spectrum includes, essentially,the entire spectrum of light emitted by the sample, rather than one or afew narrow ranges obtained by using filters. Accordingly, from a singlescan, a plurality of wavelengths may be detected as being emitted or notemitted by the sample. For example, results from a single scan of asample may be analyzed to determine whether fluorescence emitted fromthe sample has one or more of six various wavelengths.

More particularly, the microscope systems may be used in Förster (orfluorescence) resonance energy transfer (“FRET”) analysis. In FRETanalysis, the non-radiative transfer of energy from an excitedfluorescent molecule (a “donor”) to a non-excited acceptor residingnearby is analyzed. For example, a researcher may wish to determinewhether a particular ligand binds with a particular receptor of a plasmamembrane. The researcher may tag the receptors, such as G proteincoupled receptors (“GPCRs”), with fluorescent markers (e.g., yellowtags) and tag ligands with other fluorescent markers (e.g., red tags).If the receptors and ligands bind, the excited donor does not alwaysemit a yellow photon, but sometimes transfers its excitation to a nearbyacceptor, which then emits a red photon; hence, the combination of thetwo colors. Therefore, a user may desire to detect more than a singlewavelength (color) emitted by the sample, so as to be able todiscriminate between the amount of donor (e.g., yellow) and acceptor(e.g., red) signals. In this way, one may detect possible interactionsbetween the donor-receptor and the acceptor-tagged ligand, or betweentwo different receptors or any other two macromolecules. Additionally,as samples may be highly dynamic, fast acquisition of the image may bedesired to reduce the effects of time on the sample over the course ofscanning the sample. Using the microscopes system 100, multiplewavelengths of the sample's emitted light may be captured and identifiedfrom a single, high-speed scan.

FIGS. 18A-B illustrate techniques for sorting and trapping cells to bescanned using the microscope system 100. FIG. 18A illustrates an opticaltweezers technique using a tray 500 including channels 502 for sortingand trapping cells 504. A flow of an outer medium containing the cellsis fed through the channels. The cells 504 are then trapped by opticaltweezers (not shown) in their respective channels 502 for scanning. The“optical tweezers” use a highly focused laser beam to provide a verysmall attractive or repulsive force to physically hold the cells 504 inposition.

FIG. 18B illustrates a suction technique using a tray 510 includingchannels 512 for sorting and trapping the cells 504. Each channel 512includes a suction path 516 to which a slight suction is applied.Accordingly, as the cells 504 flow through the channels 512, the suctionthrough suction path 516 traps the cells 504 in their respectivechannels 512 for scanning.

Both FIGS. 18A-B result in sorted and trapped cells 504. While the cells504 are trapped, the microscope system 100 is able to perform a scan ofthe cells 504. Additionally, while trapped, the composition of the outermedium flowing through the channels 502, 512 may be altered, forinstance, by adding ligands or another chemical agent. Accordingly, thecells 504 may be subject to various treatments and analyzed using amicro-analytical assay, such as spectrally resolved fluorescencemicroscopy and FRET. The trapped cells 504 may be observed for longperiods of time while nutrients are continuously supplied through thechannels 502. In one example, this observation allows one to determinethe location at which proteins are assembled into complexes and tomonitor the proteins transport to/from the plasma membrane in theprocess of membrane recycling. In another example, the trapped cells504, expressing proteins of interests, may be presented with variableamounts of natural and artificial ligands (including drugs). In thisexample, the effect of the ligand on receptor oligomerization or on thecell in general can be investigated, in vivo. The number of channels502, 512 of the trays 500, 510 may be increased or decreased, dependingon the field of view of the microscope, the size of the cells, and otherfactors.

FIG. 19 depicts a method 550 of analyzing one or more samples to detectemissions of one or more particular wavelengths as a result of a scanusing microscope system 100. Unless noted otherwise, reference tomicroscope 100 is intended to refer to the various microscopeembodiments described herein, such as microscopes 130, 200, 250, 350,370, and 380. Additionally, although the method is described as beingimplemented with microscopy system 100, the method 550 may also becarried out using other devices.

In step 552, a sample is positioned for scanning by the microscopesystem 100. For instance, cells 504 may be introduced into channels 502,512 as depicted in FIGS. 18A-B. In step 554, the sample is subjected tochemical agents. For instance, an outer medium may be introduced intothe channels 502, 512 to tag the cells 504. In some embodiments, step554 may be bypassed if the cell is to be investigated without beingsubjected to chemical agents or performed before positioning the samplefor scanning in step 552.

In step 556, with reference to FIGS. 1 and 20, the microscope system 100is used to perform a fast, wide-area scan of the sample. For a fast,wide-area scan, a low magnification may be used to project an entireslide width onto the detector 116 (e.g., 2500 pixels wide) from a singlescan. In FIG. 20, a first portion 580 of one of the cells 504 is scannedusing a wide-area scan. Additionally, the fast, wide-area scan may beimplemented using the binning technique described with respect to FIG.17B to further improve the speed. In step 558, the imaging module 118 ofthe microscope system 100 generates one or more spectrally resolvedimages based on the pixel data obtained from the detector 116. The pixeldata may be used to generate several images of the sample, eachdepicting a different wavelength range of the fluorescence emitted fromthe sample. An exemplary image reconstruction technique to generate animage based on pixel data produced by the microscope system 100 isdescribed in U.S. Pat. No. 7,973,927, the description of which is herebyincorporated by reference.

For example, to obtain the fluorescence emission image of the sample fora particular wavelength (λn) in the case of a beam line scan, the pixelrow from the pixel data corresponding to that particular wavelength λnis extracted from each image. Each such row corresponds to a uniquey-position of the sample, and the extracted rows are reassembledaccordingly to generate an image of the sample at the wavelength λn. Inthe case of a multi-beam implementation where multiple spectra arereceived by the detector simultaneously, or a binning technique whereinmore than one pixel in a particular column corresponds to a particularwavelength band of interest λn, multiple pixel rows may be extractedfrom a single image to form the image of the sample at wavelength λn

Returning to FIG. 19, in step 560, the analysis module 120 analyzes theimage(s) or pixel data to determine whether the sample emittedfluorescence with predetermined characteristics (e.g., at one or moreparticular wavelengths, at a particular intensity, and over aparticularly sized area). In step 562, the controller 102 determineswhether the analysis module 120 identified one or more portions of thesample that emitted fluorescence with predetermined characteristics. Ifno portion is identified, as determined in step 562, the method 550returns to step 552 to begin analysis of a new sample. If a portion ofthe sample is identified, as determined in step 562, the method 550proceeds to step 564.

In step 564, a second portion 582 of the sample is scanned. The secondportion 582 is a subset of the first portion scanned in step 556, andcorresponds to the portion identified in step 582. The microscope system100 then performs a high resolution, focused scan of the second portion582. For instance, the second portion 582 is scanned without using abinning technique, or with using the binning technique of FIG. 16B. Thefocused scan may be a slower scan than the fast, wide area scan of step556. In step 566, the imaging module 118 of the microscope system 100generates one or more spectrally resolved images based on the pixel dataobtained from the detector 116.

In step 568, the analysis module 120 analyzes the image(s) or pixel datato determine whether the second portion 582 emitted fluorescence withpredetermined characteristics. In step 570, the controller 102determines whether the analysis module 120 identified the second portion582 of the sample as having emitted fluorescence with predeterminedcharacteristics. In step 572, if the second portion 582 emittedfluorescence with predetermined characteristics, the controller 102 mayperform a notifying action, such as outputting the resulting image(s) tothe user I/O 106, storing the image(s) with a flag set, generating analert or alarm, transmitting the image(s) to remote devices (e.g.,personal computers, smart phones, etc.), or take another action tonotify or highlight the sample or image(s). An alert may include one ormore of a local audible or visual message via the user I/O 106, or anaudible or visual message transmitted to a remote device (e.g., viaemail, text message, automated voice message, etc.).

In step 574, the controller 102 determines whether an additional portionwas identified in step 562. If so, the controller 102 proceeds back tostep 564 to perform a focused scan and analysis of that portion (e.g.,third portion 584). If no additional portions were identified in step562, the method 550 returns to step 552 to begin analysis of a newsample. The method 550 may repeat until no further samples are availablefor scanning.

In some embodiments, the generation steps 558 and 566 may includeexporting pixel data to a memory 104 or controller 102 for analysis bythe analysis module 120 without actually generating an image viewable bya person. Rather, the pixel data is merely received, stored, and/orarranged such that the analysis module 120 may sift through the pixeldata to determine whether the pixel data indicates that a portion of thesample emitted fluorescence with predetermined characteristics.

The higher resolution, focused scan of step 564 assists in removingfalse positives generated by the fast, wide-area scan of step 556.Accordingly, those samples identified in step 562 may be referred to astentatively positive samples, and those samples identified in step 570may be referred to as positive samples. For example, the predeterminedcharacteristics of a tentatively positive sample, as identified in step562, may include two or more markers. Additionally, the focused scan ofstep 564 results in higher resolution images that may be used to providemore detail (including morphological) of the pertinent portion of thetentatively positive sample to confirm the makeup of the portion (e.g.,the identity of a particular cell). The higher resolution images mayalso be used for later human review and analysis.

A user may store the predetermined characteristics of the fluorescenceto be detected in the memory 104 using the user I/O 106, and then mayinitiate the method 550. Once initiated, the method 550 may be anautomated process such that a plurality of samples may be scanned andanalyzed, and those samples having particular characteristics may beidentified without further user interaction.

Although the microscope system 100 has generally been described asgenerating images with a spatial (x) dimension and a wavelength (y)dimension of the sample, as noted above, multiple images obtained may bere-constructed via the imaging module 118 to form a series of images,one for each desired frequency range, with a spatial (x) dimension andspatial (y) dimension. See, for example, the image reconstructiontechniques described in U.S. Pat. No. 7,973,927. The microscope system100 may also scan the sample 132 at various depths to produce anadditional spatial (z) dimension to the images. Accordingly, 3-D images(x-y-z dimensions) may be generated for each desired frequency range bystacking multiple 2-D (x-y dimension) images generated by theabove-noted image reconstruction. In some applications, this 3-Dscanning capability allows a specific volume of samples to be scannedmuch faster than distributing that volume amongst multiple slides forscanning each at a single height in 2-D. Furthermore, samples may bescanned over time to produce a fourth (time) dimension. For example, 2-Dand 3-D images generated at time (t)=0, 1, . . . , n, may be streamed inseries to show the sample changing over the time period 0 to n.

The microscope system 100 and method 550 enable the rapid identificationof rare cells. The microscope system 100 is operable to rapidly scansmears of blood or enriched cells on microscope slides to find andidentify rare, differentially stained cells in an overwhelmingbackground of non-target cells. An exemplary rare cell is a fetal cellin maternal blood (“FCMB”), which enables the detection of geneticaberrations during the first trimester of pregnancy without risk to thefetus or mother. In most pregnancies, a few fetal cells pass theplacenta to enter the maternal blood stream, reaching concentrations of0.1 to 100 cells per milliliter of blood. The microscope system 100enables the rapid identification of these rare fetal cells against amore than million-fold excess of maternal white cells.

Several fetal cell-specific surface markers exist that allow for theirdifferential staining and identification against the background ofmaternal white blood cells. These markers may be used to enrich fetalcells using magnetic separation. The enriched cell population is thentransferred to one or more slides. After labeling with fluorescentlytagged fetal-cell-specific surface markers, the location of these cellson the slide are then identified by imaging using the microscope system100. These slides may then be deproteinized and subjected to fluorescentin situ hybridization (FISH), a process that identifies specific geneticabnormalities in a cellular genome. The microscope system 100 then scansand analyzes the portions of the (now) FISHed slides previouslyidentified to be occupied by a fetal cell. The microscope system 100then determines the absence, presence and/or multiplicity of specificfluorescent signals. The slides may be automatically loaded for analysisby the microscope system 100 using an automatic slide changer, therebyallowing continuous, fast automated scans of a plurality of samples.Additionally, the analysis of the slides may include execution of themethod 550 of FIG. 19.

A similar approach as the one described above for FCMB detection can beapplied to the identification of circulating tumor cells for earlydiagnosis of disease, or for monitoring of recurrence after therapy.Solid tumors initially arising as an organ-confined lesion eventuallyspread to distant sites through the bloodstream, generating metastasesthat are mainly responsible for their lethality. Detecting cancer cellsthat have been shed into peripheral blood provides a powerful andnoninvasive approach for diagnosing early disease and assessing theprognosis and therapeutic response. Detecting disseminated rare tumorcells in bone marrow aspirates is equally important for early diagnosis.The medical benefits of early cancer detection are significant; however,the frequency of tumor cells in these tissues is often less than 1 per10⁶ normal cells, presenting a significant problem for the diagnosingpathologist. The high sensitivity, rapid, and full-color spectralanalysis provided by the microscope system 100 enables detection offluorescently stained rare cells, such as cancer cells, to provide earlydetection of cancer cells.

The microscope system 100 may also be used to identify stem cells. TheCD34+ cell fraction of bone marrow and blood contains the hematopoieticstem cells, which are used in marrow reconstitution followingmyeloablative therapy. As the stem cells are present in small numbers,accurate quantification presents challenges. For instance, the stemcells occur at a ratio of 1 per million requires counting of 100 millioncells in order to detect the 100 target cells with a CV of 10%. Themicroscope system 100 detects specifically labeled target cells reliablyat a ratio far below this, and, accordingly, may be used to detect thehematopoietic stem cells.

Similar to the above-described rare cell detections, the microscopesystem 100 may also be used in the identification of genetic aberrationsby multi-color FISH in interphase cells for prenatal or cancercytogenetics, the rapid analysis of tissue sections afterimmuno-staining, cancer cells circulating in blood or intermixed withtissue, microbes circulating in blood, and viruses circulating in blood.Exemplary microbes include bacteria, fungi, tuberculosis (TB), malaria,and similar organisms. Exemplary viruses include human immunodeficiencyvirus (HIV) and hepatitis.

The microscope system 100 may also be used in the study of microarraysand tissue arrays, which can require a relatively large scan area inorder to allow for the analysis of thousands of individual location(“spots”). In the case of DNA arrays, each spot represents the locationof an immobilized capture probe, and, in the case of protein arrays,each spot represents either a specific antibody or a specific targetprotein. For these types of arrays, the spot size is typically about 100micron diameter, and the analysis typically involves a determination ofthe average pixel intensity per spot in one or two colors.

Tissue microarrays, in contrast, are produced using a hollow needle toremove tissue cores as small as 0.6 mm in diameter from regions ofinterest in paraffin-embedded tissues, such as clinical biopsies ortumor samples. These tissue cores are then inserted in a recipientparaffin block in a precisely spaced, array pattern. Sections from thisblock are cut using a microtome, mounted on a microscope slide, and thenanalyzed using histological analysis. Each microarray block can be cutinto 100-500 sections, which can be subjected to independent tests. Themicroscope system 100 may be employed to test the tissue microarraysusing immunohisto-chemistry and fluorescent in situ hybridization(FISH).

Analysis of tissue microarrays by the microscope system 100 isparticularly useful in analysis of cancer samples. Tissue arrays containprotein, RNA, and DNA molecules, thus providing high throughputplatforms for the rapid analysis of molecular markers associated withdisease diagnosis, prognosis, and therapeutics in patients. The analysiscan be used to validate clinical relevance of potential biologicaltargets in the development of diagnostics and therapeutics and to studynew protein markers and genes. The analysis of tissue sections generallyrequires a much higher resolution than DNA and protein arrays, and alsoincludes analysis of sub-cellular features within each section. In someembodiments, the microscope system 100 enables diffraction limitedresolution (i.e., one micron or less), and spectral resolutionselectable between 2 and 20 nm, with an acquisition speed of about twominutes per one full set of spectral images with a 15 mm×15 mm scanarea, or fourteen minutes per 22 mm×71.5 mm area, with a completewavelength spectrum.

The scanning throughput rate of the microscope system 100 may be furtheradjusted by altering the objective power (e.g., magnification of theobjective lens 152) and the stage step-size. For example, using a 40×objective, the scan area covers a 0.5 millimeter by 0.192 mm area andmay visualize about 100 cells. Using a 20× objective the scan area istwice that of the 40× objective, making about 200 cells visible.Finally, using a 10× objective, up to 600 cells may be visible.Therefore, to increase the throughput rate, a lower power objective,such as 10×, would serve to decrease scan time for each specimen,because more area is covered by a scan.

Additionally, a reduction in resolution, or reduction in over-sampling,is another way of decreasing scan time. If the stage step size isincreased, the spatial resolution is reduced, but more cells arevisualized, increasing sample size. For example, if the step size isincreased to 8 microns, the step size allows scanning of an entiremicrotiter well in a single scan field.

FIG. 21 illustrates the microscope system 100 implemented as atransmission microscope 600 for transmission imaging. As shown in FIG.21, the transmission microscope 600 includes components previouslydescribed with respect to the above-noted microscopes (e.g., microscopes130, 200, 250, 350, 370, and 380). In contrast to the above-notedmicroscopes, when the transmission microscope 600 scans light across thesample 132, the fluorescence 154 emitted from the sample 132 is receivedby a second objective lens 152 b that is on the opposite side of thesample 132 from which the beams 142 are received. Accordingly, thefluorescence 154 emitted does not go back towards the objective lens 152a from which the beams 142 came.

The other above-noted, non-descanning microscopes 200 and 250 may alsobe implemented as transmission imaging microscopes by adding a secondobjective lens 152 b as described with respect to transmissionmicroscope 600.

FIG. 22 illustrates the microscope system 100 implemented as amulti-excitation beam scanning microscope 650 for analyzing sample 132.For simplification, the controller 102, memory 104, user I/O 106, andbus 107 of FIG. 1 are not shown. As shown in FIG. 21, the microscope 650includes components previously described with respect to the above-notedmicroscopes (e.g., microscopes 130, 200, 250, 350, 370, and 380, and600).

The microscope 650 includes a multi-wavelength beam generator 652, whichoutputs a first wavelength light beam 654 a and a second wavelengthlight beam 654 b. The first and second wavelength light beams 654 a-bhaving a first and second wavelength, respectively. The light beams 654a-b are reflected by mirror 144, scanned by the scanning mirrors 146,and focused by the lens 147. The dichroic mirror 150 reflects the lightbeams 654 a-b toward the lens 149, which collimates the light beams 654a-b. The objective lens 152 focuses the light beams 654 a-b to beampoints on the sample 132. The beam points are spaced apart in thex-direction, but are at the same position in the y-dimension, as will bedescribed in further detail with respect to FIGS. 23A-D. In response tothe light beams 654 a-b, the sample 132 emits fluorescence beams 656 a-btowards the objective lens 152 and lens 149. The lens 149 focuses eachemitted fluorescence beams 656 a-b to a point on the detector 158. Thefluorescence beams 656 a-b, however, first pass through the short-passdichroic mirror 150 and a light dispersive element 160. The short-passdichroic mirror 150 allows visible light to pass through, whilereflecting most of the infrared light components of the emittedfluorescence beams 656 a-b. The dispersive element 160 disperses thelight into its spectral components to form a continuous spectrum ofvarying wavelengths that spread in the y-direction of the detector 158.The dispersed light beams 654 a-b impinge the detector as spectra at thesame y-position, but spaced apart in the x-direction (see, e.g., FIGS.24A-D).

The multi-wavelength beam generator 652 may be implemented using varioustechniques. For instance, the multi-wavelength beam generator 652 mayinclude two light sources that each emits a particular wavelength beam.Additionally, the multi-wavelength beam generator 652 may include asingle laser that has two output beams, each with a differentwavelength. Alternatively, multi-wavelength beam generator 652 mayinclude a broadband light source in conjunction with one or more offilters, gratings, prisms, dichroic mirrors, etc. to produce two lightbeams, each having a different wavelength. Using different wavelengthsto excite the sample enables the microscope 650 to provide spectralresolution in the excitation channel.

FIGS. 23A-D illustrate a multi-excitation beam scan on the sample 132.The objective lens 152 focuses the light beams 654 a and 654 b on thesample as points (a₁ and a₂, respectively). As illustrated in FIGS.23A-D, the light beams 654 a and 654 b are at the same y position on thesample 132, but are spaced apart in the x-direction. The light beams 654a and 654 b both follow the same path 658 to scan the sample and remainin lock-step (i.e., at the same distance apart along the x-dimension andat the same y-position). FIG. 23A illustrates an initial position, whileFIGS. 23B through 23D illustrate the light beams 654 a-b in variouspositions along the path 658. To enable the light beam 654 a to reachthe left-most portions of the sample, the light beam 654 b istemporarily focused on a point to the left of the scan area each timethe light beams 654 a and 654 b reach the left side of the sample 132(see, e.g., FIG. 23A). To enable the light beam 654 b to reach theright-most portions of the sample, the light beam 654 b is temporarilyfocused on a point to the right of the scan area each time the lightbeams 654 a and 654 b reach the right side of the sample 132 (see, e.g.,FIGS. 23C and 23D). Accordingly, as the light beams 654 a-b scan acrossthe sample 132 to the right, the light beam 654 b trails the light beam654 a, but as the light beams 654 a-b scan across the sample 132 to theleft, the light beam 654 a trails the light beam 654 b. Thus, the lightbeam 654 a and 654 b alternate between leading and trailing positionsalong the path 658.

FIGS. 24A-D illustrate a multi-excitation beam scan on the detector 158.As noted above, the emitted fluorescence beams 656 a and 656 b passthrough a dispersive element 160 that disperse the beams into theirspectral components, which form spectra 660 a and b. The spectra 660 aand 660 b have a line shape extending along the y-direction, and each ispositioned at the same y-position and spaced apart in the x-direction.Each spectrum 660 is formed of a continuous spread of the emittedfluorescence over a range of wavelengths. Each point along the y-axis ofthe spectrum 660 includes components of the emitted fluorescence at adifferent wavelength. For instance, the upper/top portion of thespectrum 660 may include the larger wavelengths components, while thelower/bottom portion includes the shorter wavelength components. Similarto the beams 654 a and 654 b, the spectra 660 a and 660 b generallyfollow the same the path 658. Pixel data is obtained from the detector158 as the spectra 660 a and 660 b reach each point along the scan path658, which corresponds to the beam points 654 a-b reaching each positionof the sample 132 along scan path 658.

In some embodiments, one or more additional excitation beams areincluded, each having a particular wavelength and spaced apart in thex-direction from the light beams 654 a and 654 b, but at the samey-position, on the sample 132. In some embodiments, the excitation lightbeams 654 a-b are replicated at different y-positions on the sample 132to combine the concepts of the microscope 650 with the multi-beam pointscan of the microscope 130 (FIG. 2). Accordingly, as shown in FIGS.25A-B, additional excitation light beams 662 a and 662 b are provided onthe sample 132. The light beam 662 a has the same wavelength as lightbeam 654 a, and the light beam 662 b has the same wavelength as thelight beam 654 b. The light beams 662 a and b follow a path 664, whichis similar to, but displaced in the y-direction from, the path 658. Thelight beams 662 a-b cause the sample to emit fluorescent beams, whichare dispersed by the dispersive element 160 and impinge the detector 158as spectra 666 a-b. The spectra 666 a-b follow the path 664 on thedetector 158, similar to the spectra 660 a-b following the path 658. Byintroducing the additional light beams 662 a-b, the scan time of thesample 132 may be reduced relative to the embodiments of FIG. 22.Although shown in a nondescanned implementation, the microscope 650 mayalso be a arranged in a descanned or half-descanned implementation.

In some embodiments, the various microscopes (e.g., 130, 200, 250, 350,370, 380, 600 and 650) include additional telescopes, lenses, filters,etc. to focus and transmit light between the various componentsillustrated, such as between the tube lens 156 and the detector 158.Additionally, although the sample 132 is often described above asemitting fluorescence (e.g., fluorescence 154, 210, and 256), the energyemitted by the sample 132 in response to a scan for detection by thedetector 158 may include one or more of fluorescence, elasticallyscattered light (i.e., Raman), second harmonic signals, and thirdharmonic signals, or other light types.

Thus, the invention provides, among other things, a system and method ofhigh-speed microscopy using a microscope with spectral resolution. Themicroscope may include one of a multi-beam point scanning microscope, asingle beam line scanning microscope, and a multi-beam line scanningmicroscope. The systems and methods provide improved scanning speeds,rendering the microscopes advantageous in a variety of applications,including medical research and diagnostics. Various features andadvantages of the invention are set forth in the following claims.

What is claimed is:
 1. A microscope for generating a multi-dimensional,spectrally resolved image, the microscope comprising: a light sourcethat generates a pulsed light beam; a curved mirror that reflects thepulsed light beam as a light beam line; a scanning mechanism that scansthe light beam line across a sample to cause the sample to emit energy;a dispersive element that receives the emitted energy from the sampleand disperses the emitted energy into spectral components that formcontinuous spectrum area, the continuous spectrum area corresponding tothe light beam line; and a detector that receives the continuousspectrum.
 2. The microscope of claim 1, further comprising a secondcurved mirror that reflects the pulsed light beam as a second light beamline, wherein the scanning mechanism scans the second light beam lineacross the sample to cause the sample to emit additional energy, thedispersive element receives the additional emitted energy from thesample and disperses the additional energy into second spectralcomponents that form a second continuous spectrum area, the secondcontinuous spectrum area corresponding to the second light beam line,and the detector receives the second continuous spectrum.
 3. Themicroscope of claim 1, wherein the scanning mechanism includescomputer-controlled x-y scanning mirrors positioned between the lightsource and the sample.
 4. The microscope of claim 1, wherein thescanning mechanism includes computer-controlled sample positioners thatmove the sample to cause the light beam line to scan the sample.
 5. Themicroscope of claim 1, wherein the microscope is a two-photonmicroscope.
 6. The microscope of claim 1, wherein the curved mirror is adeformable mirror shaped by control signals from a mirror controller. 7.The microscope of claim 6, wherein the shape of the deformable mirror isadjusted based on feedback from a light sensor related to aberrations inthe pulsed light beam.
 8. The microscope of claim 1, further comprisingan imaging module for receiving pixel data output by the detector andgenerating one or more images based on the pixel data.
 9. The microscopeof claim 1, wherein the emitted energy is one of fluorescence,elastically scattered light, second harmonic signals, and third harmonicsignals.
 10. The microscope of claim 1, further comprising a firstobjective positioned between the scanning mechanism and the sample,wherein the first objective receives the light beam line from thescanning mechanism and transmits the light beam line to the sample, anda second objective positioned between the sample and the dispersiveelement, wherein the second objective receives the emitted energy fromthe sample and transmits the emitted energy to the dispersive element.11. A microscope for generating a multi-dimensional, spectrally resolvedimage, the microscope comprising: a light source that generates a pulsedlight beam; a multi-beam generator that receives the pulsed light beamand emits multiple light beams; a light beam line generator thatreceives the multiple light beams and emits multiple light beam lines; ascanning mechanism that simultaneously scans the multiple light beamlines across a sample to cause the sample to emit energy; a dispersiveelement that receives the emitted energy from the sample and dispersesthe emitted energy into spectral components that form continuousspectrum areas, each continuous spectrum area corresponding to one ofthe multiple light beam lines; and a detector that receives eachcontinuous spectrum simultaneously.
 12. The microscope of claim 11,wherein the multi-beam generator is a grating.
 13. The microscope ofclaim 11, wherein the multi-beam generator is one of a beam splitter andan array of lenses.
 14. The microscope of claim 11, wherein the lightbeam line generator includes a first curved mirror and a second curvedmirror.
 15. The microscope of claim 11, wherein the light beam linegenerator includes a first lens and a second lens.
 16. The microscope ofclaim 11, wherein the emitted energy is one of fluorescence, elasticallyscattered light, second harmonic signals, and third harmonic signals.17. The microscope of claim 11, further comprising a first objectivepositioned between the scanning mechanism and the sample, wherein thefirst objective receives the light beam lines from the scanningmechanism and transmits the light beam lines to the sample, and a secondobjective positioned between the sample and the dispersive element,wherein the second objective receives the emitted energy from the sampleand transmits the emitted energy to the dispersive element.
 18. Amicroscope for generating a multi-dimensional, spectrally resolvedimage, the microscope comprising: a light source that generates a pulsedlight beam; a multi-beam generator that receives the pulsed light beamand emits multiple light beams; a scanning mechanism that simultaneouslyscans the multiple light beams across a sample, the sample emittingenergy corresponding to each of the multiple light beams; a dispersiveelement that receives the emitted energy from the sample and dispersesthe emitted energy into spectral components that form continuousspectrum lines, each continuous spectrum line corresponding to one ofthe multiple light beams; and a detector that receives each continuousspectrum line simultaneously.
 19. The microscope of claim 18, whereinthe multi-beam generator is a grating.
 20. The microscope of claim 18,wherein the scanning mechanism includes one of computer-controlled x-yscanning mirrors positioned between the light source and the sample, andcomputer-controlled sample positioners that move the sample to cause themultiple light beams to scan the sample.
 21. The microscope of claim 18,wherein the microscope is a two-photon microscope.
 22. The microscope ofclaim 18, wherein the emitted energy is one of fluorescence, elasticallyscattered light, second harmonic signals, and third harmonic signals.23. The microscope of claim 18, further comprising a first objectivepositioned between the scanning mechanism and the sample, wherein thefirst objective receives the light beams from the scanning mechanism andtransmits the light beams as light beam points to the sample, and asecond objective positioned between the sample and the dispersiveelement, wherein the second objective receives the emitted energy fromthe sample and transmits the emitted energy to the dispersive element.24. The microscope of claim 18, wherein the multiple light beams includeat least a first light beam with a first wavelength, and a second lightbeam having a second wavelength.
 25. The microscope of claim 24, whereinthe first light beam follows a path when scanned across the sample andthe second light beam follows the path when scanned across the sample.26. The microscope of claim 24, wherein the first light beam and thesecond light beam alternate between leading and trailing positions alongthe path.
 27. The microscope of claim 24, wherein the first light beamis scanned across the sample in an x-direction at a y-position, and thesecond light beam is simultaneously scanned across the sample in thex-direction at the y-position, the second light beam being offset in thex-direction from the first light beam.
 28. The microscope of claim 24,wherein the continuous spectrum line for the first light beam follows apath across the detector during the scan, and the continuous spectrumline for the second light beam follows the path across the detectorduring the scan.
 29. The microscope of claim 24, wherein the continuousspectrum line for the first light beam and the continuous spectrum linefor the second light beam alternate between leading and trailingpositions along the path.
 30. A method of generating amulti-dimensional, spectrally resolved image, the method comprising:generating a pulsed light beam; reflecting, by a curved mirror, thepulsed light beam as a light beam line; scanning the light beam across asample to cause the sample to emit energy; dispersing the emitted energyfrom the sample into spectral components that form a continuous spectrumarea, the continuous spectrum area corresponding to each point along thelight beam line; and receiving, by a detector, the continuous spectrum.31. The method of claim 30, further comprising reflecting, via a secondcurved mirror, the pulsed light beam as a second light beam line,scanning the second light beam line across the sample to cause thesample to emit additional energy, dispersing the additional energy intosecond spectral components that form a second continuous spectrum area,the second continuous spectrum area corresponding to the second lightbeam line, and receiving, by the detector, the second continuousspectrum.
 32. The method of claim 30, further comprising, outputting, bythe detector, pixel data resulting from receiving the continuousspectrum and the second continuous spectrum; receiving, by an imagingmodule, the pixel data; and generating, via an imaging module one ormore images based on the pixel data.
 33. A method of generating amulti-dimensional, spectrally resolved image, the method comprising:generating a pulsed light beam; receiving, via a multi-beam generator,the pulsed light beam; emitting, by the multi-beam generator, multiplelight beams in response to the pulsed light beam; simultaneouslyscanning the multiple light beams across a sample, the sample emittingenergy corresponding to each of the multiple light beams; dispersing theemitted energy into spectral components that form continuous spectrumlines, each continuous spectrum line corresponding to one of themultiple light beams; and receiving, by a detector, each continuousspectrum line simultaneously.
 34. The method of claim 33, furthercomprising, outputting, by the detector, pixel data resulting fromreceiving the continuous spectrum lines; receiving, by an imagingmodule, the pixel data; and generating, via the imaging module one ormore images based on the pixel data.